Reliable pseudo-labeling prediction framework for new event type induction

doi:10. 19682 / j. cnki. 1005-8885. 2023. 0009

中国邮电高校学报(英文) ›› 2023, Vol. 30 ›› Issue (5): 42-50.doi: 10. 19682 / j. cnki. 1005-8885. 2023. 0009

所属专题： Special Topic on Digital Human

• Special Topic : Digital Human • 上一篇下一篇

Reliable pseudo-labeling prediction framework for new event type induction

杨琪¹,徐雅静¹,吕远²,肖波¹,陈光¹

北京邮电大学

收稿日期:2022-10-19 修回日期:2023-03-06 出版日期:2023-10-31 发布日期:2023-10-30
通讯作者: 徐雅静 E-mail:xyj@bupt.edu.cn
基金资助:
the National Natural Science Foundation of China (62076031).

Reliable pseudo-labeling prediction framework for new event type induction

School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing 100876, China

Received:2022-10-19 Revised:2023-03-06 Online:2023-10-31 Published:2023-10-30
Contact: Ya-Jing XU E-mail:xyj@bupt.edu.cn
Supported by:
the National Natural Science Foundation of China (62076031).

摘要/Abstract

摘要：

As a subtask of open domain event extraction ( ODEE), new event type induction aims to discover a set of unseen event types from a given corpus. Existing methods mostly adopt semi-supervised or unsupervised learning to achieve the goal, which uses complex and different objective functions for labeled and unlabeled data respectively. In order to unify and simplify objective functions, a reliable pseudo-labeling prediction (RPP) framework for new event type induction was proposed. The framework introduces a double label reassignment ( DLR) strategy for unlabeled data based on swap-prediction. DLR strategy can alleviate the model degeneration caused by swap-predication and further combine the real distribution over unseen event types to produce more reliable pseudo labels for unlabeled data. The generated reliable pseudo labels help the overall model be optimized by a unified and simple objective. Experiments show that RPP framework outperforms the state-of-the-art on the benchmark.

关键词: open domain, event type induction, pseudo label, unified objective, swap-predication

Abstract:

Key words: open domain, event type induction, pseudo label, unified objective, swap-predication

参考文献

[1] LIU X, HUANG H Y, ZHANG Y. Open domain event extraction using neural latent variable models. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, Jul 28 - Aug 2, Florence, Italy. Stroudsburg, PA, USA:

Association for Computational Linguistics, 2019: 2860 -2871.

[2] HUANG L F, JI H. Semi-supervised new event type induction and event detection. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing ( EMNLP'20), 2020, Nov 16 - 20, Punta Cana, Dominican. Stroudsburg, PA, USA: Association for Computational Linguistics, 2020: 718 -724.

[3] SHEN J M, ZHANG Y Y, JI H, et al. Corpus-based open-domain event type induction. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP'21), 2021, Nov 7 - 11, Punta Cana, Dominican. Stroudsburg, PA, USA: Association for Computational Linguistics, 2021: 5427 -5440.

[4] CARON M, MISRA I, MAIRAL J, et al. Unsupervised learningof visual features by contrasting cluster assignments. Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS'20), 2020, Dec 6 - 12, Vancouver, Canada. Red Hook, NY, USA: Curran Associates Inc, 2020: 9912 -9924.

[5] FINI E, SANGINETO E, LATHUILIERE S, et al. A unified objective for novel class dIscovery. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision ( ICCV'21), 2021, Oct 10 - 17, Montreal, Canada. Piscataway, NJ, USA: IEEE, 2021: 9264 -9272.

[6] ASANO Y M, RUPPRECHT C, VEDALDI A. Self-labelling viaimultaneous clustering and representation learning. Proceedings of the 8th International Conference on Learning Representations (ICLR'20), 2020, Apr 26 -30, Addis Ababa, Ethiopia. 2020: 1 -22.

[7] CHEN Y B, XU L H, LIU K, et. al. Event extraction via dynamic multi-pooling convolutional neural networks. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing: Vol 1 (Long Papers), 2015, Jul 26 - 31, Beijing, China. Stroudsburg, PA, USA: Association for Computational Linguistics, 2015: 167 -176.

[8] LIN Y, JI H, HUANG F, et. al. A joint neural model for information extraction with global features. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Vol 1 (Long Papers), 2020, Jul 5 - 10, Seattle, WA, USA. Stroudsburg, PA, USA: Association for Computational Linguistics, 2020: 7999 -8009.

[9] WANG Z Q, WANG X Z, HAN X, et al. CLEVE: contrastive pre-training for event extraction. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: Vol 1 (Long Papers), 2021, Aug 1 - 6, Bangkok, Thailand. Stroudsburg, PA, USA: Association for Computational Linguistics, 2021: 6283 -6297.

[10] HUANG L F, JI H, CHO K, et al. Zero-shot transfer learning for event extraction. Proceedings of the 56th Annual Meeting of the

Association for Computational Linguistics: Vol 1 (Long Papers), 2018, Jul 15 - 20, Melbourne, Australia. Stroudsburg, PA, USA: Association for Computational Linguistics, 2018: 2160 -2170.

[11] ACE ( Automatic Content Extraction ) English annotation guidelines for events, Version 5. 4. 3. Philadelphia, PA, USA: Linguistic Data Consortium, 2005.

[12] CHAMBERS N. Event schema induction with a probabilistic entity-driven model. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, 2013, Oct 18 -21, Seattle, WA, USA. Stroudsburg, PA, USA: Association for Computational Linguistics, 2013: 1797 -1807.

[13] NGUYEN K, TANNIER X, FERRET O, et al. Generative event schema induction with entity disambiguation. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing: Vol 1 (Long Papers), 2015, Jul 26 - 31, Beijing, China. Stroudsburg, PA, USA: Association for Computational Linguistics, 2015: 188 -197.

[14] YUAN Q, REN X, HE W Q, et al. Open-schema event profiling for massive news corpora. Proceedings of the 27th ACM International Conference on Information and Knowledge Management, 2018, Oct 22 – 26, Torino, Italy. New York, NY, USA: ACM, 2018: 587 -596.

[15] LAI V D, NGUYEN T H. Extending event detection to new types with learning from keywords. Proceedings of the 5th Workshop on Noisy User-generated Text ( W-NUT'19), 2019, Nov 4, Hong Kong, China. Stroudsburg, PA, USA: Association for Computational Linguistics, 2019: 243 -248.

[16] HUANG L F, CASSIDY T, FENG X C, et al. Liberal event extraction and event schema induction. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics: Vol 1 ( Long Papers), 2016, Aug 7 - 12, Berlin, Germany. Stroudsburg, PA, USA: Association for Computational Linguistics, 2016: 258 -268.

[17] LIANG X B, WU L J, LI J T, et al. R-Drop: regularized dropout for neural networks. Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS'21), 2021, Dec 6 -14, Sydney, Australia. Red Hook, NY, USA: Curran Associates Inc, 2021: 1 -21.

[18] CUTURI M. Sinkhorn distances: lightspeed computation of optimal transport. Proceedings of the 26th Conference on Neural Information Processing Systems ( NeurIPS'13 ): Vol 2, 2013, Dec 5 -10, Lake Tahoe, NV, USA. Red Hook, NY, USA: Curran Associates Inc, 2013: 2292 -2300.

[19] WAGSTAFF K, CARDIE C, ROGERS S, et al. Constrained K-means clustering with background knowledge. Proceedings of the 18th International Conference on Machine Learning (ICML'01), 2001, Jun 28 - Jul 1, Williamstown, MA, USA. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc, 2001: 577 -584.

[20] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies ( NAACL-HLT'19 ): Vol 1 ( Long and Short Papers), 2019, Jun 2 - 7, Minneapolis, MN, USA. Stroudsburg, PA, USA: Association for Computational Linguistics, 2019: 4171 -4186.

Reliable pseudo-labeling prediction framework for new event type induction

Reliable pseudo-labeling prediction framework for new event type induction

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 0

编辑推荐

Metrics

本文评价